Method and apparatus for detecting and displaying a representative image of a shot of short duration in a moving image

ABSTRACT

A method and apparatus for displaying a representative image corresponding to a shot of a duration within a specified range included in a plurality of shots of a moving image. The moving image includes a plurality of successive frames each having an image and each shot includes a plurality of the successive frames. In the invention a plurality of shot change points are sequentially detected from the moving image. The shot change points represent points between a pair of frames where images of a preceding shot and images of a succeeding shot have changed. Thereafter a detection is performed to determine whether a duration of a shot between a preceding shot change point and a succeeding shot change point is within the specified range. A representative image corresponding to the shot having a duration within the specified range is displayed along with duration time information representing the duration of the shot. A reduced image from the shot can be displayed as the representative image and the number of detected shots having a duration within the specified range can be reduced according to preset conditions.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related to application Ser. No. 08/323,866, filedOct. 17, 1994 and application Ser. No. 08/604,606, filed Feb. 21, 1996,the disclosures of each being incorporated herein by reference.

BACKGROUND OF THE INVENTION

The present invention relates to inspecting the quality of moving imagessuch as videos or motion pictures. More particularly, the presentinvention relates to a method and apparatus for detecting a shot ofshort time duration inserted in a moving image stored on video tape or avideo disc, and displaying a representative image of the shot of shorttime duration.

Due to the popularity of satellite broadcasting, cable television andthe like, in addition to existing television broadcasting, the number ofbroadcasting channels have steadily increased. If a wide-bandcommunications infrastructure called an "Information Highway" isestablished in the future, broadcasting will be easy to carry out. Thus,far more industrialists will take part in the broadcasting industry thando today further accelerating the increase in the number of broadcastingchannels. With such a large amount of information being broadcasted itis possible that some of the information may contain illegal imageswhich are intended to manipulate people. For example, there is a knownphenomena called "Subliminal Suggestion" which is intended for illegalpsychological manipulation of an audience. Subliminal suggestion isaccomplished by inserting special images of extremely short timeduration into a moving image. It is thought that subliminal suggestionis capable of subconsciously influencing an audience to perform asuggested act without the audience being aware that such a suggestionhas occurred. For this reason, it is necessary to determine whether eachprogram to be broadcast contains subliminal suggestion information andprevent broadcasting of such program. However, according to conventionaltechnology, it is necessary to manually inspect a moving image whilefeeding it on a frame-by-frame basis. Thus, even if the moving image canbe inspected at a rate of one frame per second, at least 48 hours wouldbe required to inspect an entire 2-hour motion picture. As a result, itis physically impossible to inspect every program to be broadcast.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a method and apparatusfor detecting a shot of short time duration inserted in a moving imagestored on video tape or a video disc and displaying a representationimage of the shot.

To achieve the above object, the present invention provides inputapparatus for inputting a moving image as a target sequentially in timeon a frame-by-frame basis (one frame is one still image whichconstitutes part of a moving image, and generally televisions arearranged to display 30 frames per second), a shot change point detectorfor sequentially detecting from the moving image a plurality of shotchange points representing points between a pair of frames where imagesof a preceding shot and images of a succeeding shot have changed, a shotduration detector for detecting a time duration of a shot between a timeof a preceding shot change point (hereinafter referred to as the firsttime instant) and a time of a succeeding shot change point (hereinafterreferred to as the second time instant), and apparatus for displaying arepresentative image of the shot and the time duration of the shot, ifthe duration time of the shot between the first and second time instantsis shorter than a set time period. The time of occurrence of the shotcan also be displayed with the representative image and time durationthereof.

There are also provided apparatus for displaying the representativeimage in the form of a list and apparatus for displaying the list so asto be indicative of the time duration of the shot corresponding to therepresentative image. Thus, the time duration of the shot represented bythe representative image can be easily checked by viewing.

Further provided is apparatus for storing the first or any one of theframes of the shot as the representative image when the duration of theshot is being detected by the shot duration detector. The stored framemay be later retrieved as the representative image when the duration ofthe shot is shorter than the set time period. Also provided is apparatusfor reducing the stored frame, retrieving the reduced frame as therepresentative image when the duration of the shot is shorter then theset time, and displaying the reduced frame as an icon representative ofthe shot of short time duration.

According to the present invention, it is possible to automaticallydetect a shot of short time duration without relying on human labor,thereby making it possible to automatically detect a picture portion inwhich an illegal image may be inserted. Since it is similarly possibleto detect a meaningless shot of short duration which has beenerroneously inserted during editing, it is also possible to improve thequality control of broadcast programs by detecting low quality shortduration frames and deleting them the same as illegal images.

In the present invention the number of detected shots of short timeduration to be inspected by a user can be reduced according to twoconditions. The first condition is whether a plurality of shots of shortduration have been successively detected at intervals of a durationshorter than a specified duration. The second condition is whether ashot of short time duration has been detected within a short period oftime after detection of a previous shot of short time duration andwhether the image of the detected shot of short time duration is similarto the image of the previously detected shot of short time duration.

If the first condition is satisfied all of the detected shots of shorttime duration are grouped together in a group with a singlerepresentative image. If the second condition is satisfied the detectedshot of short time duration is added to the group containing thepreviously detected shot of short time duration.

In the present invention a representative image of a group of detectedshots of short time duration is displayed in such a manner to indicatethat it represents a group of detected shots of short time duration.Selection by the user of such a representative image causes thehierarchical display of the representative image of the group andrepresentative images of all of the detected shots of short timeduration that make up the group. Selection of any of the representativeimages of the detected shots of short time duration that make up thegroup allows the user to individually inspect the images of the detectedshots of short time duration that make up the group.

Thus, by use of the present invention since representative imagescorresponding to a detected doubtful picture portion which may beillegal are displayed in the form of a list, the detected doubtfulpicture portion can readily be checked. If there is an error in thepicture portion, the user can quickly check the picture portion whileviewing it with his/her own eyes.

Further, since time information indicating the duration and when such apicture portion was detected is also displayed together with thecorresponding representative image, it is possible to readily know notonly the position of the picture portion in the entire picture but alsothe time duration of the picture portion, variations in the distributionof the picture portion with time, and the like.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be more apparent from the following detaileddescription, when taken in conjunction with the accompanying drawings,in which:

FIG. 1 is a block diagram of an embodiment of the present invention;

FIG. 2 is a view illustrating the concept of illegal image insertion;

FIG. 3 is a flowchart of a computer program for detecting a shot ofshort time duration;

FIG. 4 is an example of an interface screen for detecting a shot ofshort time duration;

FIG. 5 is a view illustrating an example of the procedure of displayinga detected shot of short time duration in the form of a list;

FIG. 6 is a view illustrating another example of the procedure ofdisplaying a detected shot of short time duration in the form of a list;

FIG. 7 is a view illustrating an example of the procedure of displayinga group of detected shots of short time duration in the form of ahierarchical list;

FIG. 8 is a flowchart of a computer program for detecting shots of shorttime duration according to two conditions; and

FIG. 9 is a schematic diagram illustrating the structure of a storagearea containing data of a group of detected shots of short timeduration.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

An embodiment of the present invention will be described below in detailin reference to FIGS. 1-9.

FIG. 1 is one example of a schematic block diagram of a systemconstruction for realizing the present invention. Reference numeral 1denotes a display device such as a CRT for displaying an output imagefrom a computer 4. Instructions can be given to the computer 4 throughan input device such as a keyboard or a pointing device. A moving-imagereproducing device 10 is a tuner for receiving broadcast programs fromground-wave broadcasting, satellite broadcasting, cable television orthe like, or a device for reproducing a moving-image recorded on anoptical disc, video tape or the like. A video signal outputted from themoving-image reproducing device 10 is sequentially converted intodigital image data by an A/D converter 3, and the digital image data issent to the computer 4. In the computer 4, the digital image data isstored in a memory 9 via an interface 8, and is processed by a CPU 7 inaccordance with a program stored in the memory 9.

The interface 8 can be a video capture board such as the FPS 60 videocapture board produced by Fast Electronic U.S., Inc. or the videocapture boards described in "Byte's Video Workshop", by S. Diehl, Byte,May 1995, pp. 97-104, if the moving-image reproducing device 10 is astandard analog video tape player. However, such a video capture boardis not necessary if the moving-image reproducing device 10 is a digitalvideo player. Also the A/D converter 3 would not be necessary. Ifindividual frames of the moving image handled by the moving-imagereproducing device 10 are respectively assigned numbers (frame numbers)in ascending order from the leading frame of the moving image, a movingimage corresponding to a particular scene can be called and reproducedby sending the corresponding frame number to the moving-imagereproducing device. Various kinds of information may also be stored inan external information storage device 6, if needed for processing.Various kinds of data which are prepared by processing to be describedbelow are stored in the memory 9, and are referred to, as required.

FIG. 2 is a schematic view of frame images which are ordered in time ina portion of the moving image in which a typical subliminal image isinserted. A frame image 200 is the inserted image. Since shot changepoints are present before and after the frame image 200, the frame image200 can be regarded as a shot having a shot time duration of one frame.The following is a detailed description of a method of detecting a shothaving such a short time duration.

FIG. 3 is one example of a flowchart of a short-time shot detectingcomputer program which is executed by the computer 4 shown in FIG. 1.The program is stored in the memory 9. First of all, as initializingprocessing, the CPU 7 sets the various kinds of variables required forexecution of the program to initial values (step 300). The initializingprocessing also includes the initialization of a time Tlast of theprevious shot change point (step 302). Next the CPU 7 accepts a frameimage fn which is outputted from the moving-image reproducing device 10(step 304). Thereafter, the CPU 7 determines whether there is a shotchange point between the frame image fn and the previous frame fn-1(step 306).

As a method of detecting such a shot change point, it is possible touse, for example, the method disclosed in "AUTOMATIC INDEXING METHOD ANDOBJECT SEARCHING METHOD FOR COLOR PICTURE," Journal of InformationProcessing Society of Japan, Vol. 33, No. 4, the method disclosed inJapanese Patent Laid-Open No. 111181/1992 which corresponds to U.S. Pat.No. 5,083,860 or the method disclosed in application Ser. No.08/604,606, filed Feb. 21, 1996. In Japanese Patent Laid-Open No.111181/1992, basically, if a significant difference in image between twosuccessive frames is observed in a picture, it is determined that thereis a shot change point between the two frames.

If it is not determined that there is a shot change point, the processreturns to step 304, in which the frame image is accepted (step 308). Ifit is determined that there is a shot change point, the value of Tcur isreplaced with the current time, and the difference between the value ofTcur and the time Tlast of the previous shot change point is calculatedto obtain a time duration Tdur of the shot. Then, the value of Tlast isreplaced with the value of Tcur to prepare for the next time where steps304-308 are performed (step 310).

The time of the shot change point may be the time of a clock, or mayalso be a frame number or a time code indicative of the position of ashot change point on tape. In this case, at the same time that a shotchange point is detected, the CPU 7 obtains the corresponding framenumber or time code of the moving-image reproducing device 10. Ofcourse, in a case where LTC or VITC time code is used, data which issent via a signal line in synchronism with a frame image may be read.Then, the contents of a two-dimensional array I2 is copied to atwo-dimensional array I1 for storing an icon image to be used as arepresentative image (step 312). After that, the frame image fn isreduced, as required, and copied to the array I2 (step 314).

Since the time period required for this copy processing is considerablylong, the arrays I1 and I2 may be managed with address pointers relativeto their respective data areas, and the process of step 312 may alsoreplaced with the processing of merely exchanging the address pointers.Then, it is determined whether the previously obtained time durationTdur is smaller than a threshold Th1 (step 316). If the value of Tdur isgreater than the threshold Th1, the process returns to Step 304. If thevalue of Tdur is smaller than the threshold Th1, the icon image whichhas been stored in the array I1 as the representative image is displayedon the display device 1 (step 318).

The time duration of each shot is identified after the shot has beenswitched to the next shot, with the result that if a representativeimage is accepted when the time duration of a particular shot isidentified, an image corresponding to another shot will be accepted.However, by storing a representative image corresponding to the previousshot change point in the above-described manner, it is possible toaccurately capture a representative image of a shot of short timeduration. In addition, the time obtained by subtracting the value ofTdur from the value of Tcur is displayed together as the start time ofthe shot (step 320). Further, the value of Tdur itself or, as will bedescribed later, a symbol which varies according to the value issimultaneously displayed (step 322). Those representative images, thevalue of Tcur-Tdur and the value of Tdur are stored in the memory 9 ofthe computer 4 or the external storage device 6 so that they arereferred to, as in the case of a list display. Finally, an alarm issounded to warn a user of the presence of the short time direction shot,as required (step 324). Then, the process returns to Step 304 to repeatthe above-described processings.

FIG. 4 shows one example of a display screen of the system whichexecutes the program shown in FIG. 3. The display screen can begenerated using a multi-window function such as that provided by theWINDOWS operating system. Reference numeral 400 denotes a cursor whichcan be made to point to an arbitrary position on the screen by use of apointing device 5 such as a mouse. However, the cursor 400 is not neededif an arbitrary point can be directly pointed to by using a touch panelor other such devices. Reference numeral 402 denotes a preview screen ofa picture. Reference numeral 404 denotes a control panel on whichbuttons similar to those of video tape recorder (VTR) are arranged, andby pointing to any of the buttons on the control panel 404, a controlinstruction such as PLAYBACK or FAST FORWARD FEED can be sent to themoving-image reproducing device 10 connected to the computer 4.

A reproduced picture can be checked through the preview screen 402.Reference numeral 406 denotes a screen area for displaying, in the formof a list, icons 408 indicative of representative images whichrespectively correspond to detected shots of short time duration. If amultiplicity of icons are to be displayed, any of them, which are out ofthe screen area 406, may be displayed by pointing to a scroll bar 410.Since a scroll interface itself is a well known interface of "Windows"of Microsoft Corp. and the "Macintosh Computer" of Apple Computers, nodetailed description is given herein.

If any of the icons 408 is pointed to, the moving-image reproducingdevice 10 is controlled to cue and retrieve a shot which corresponds tothe pointed to icon. The cueing is performed based on the information(the above-described value of Tcur-Tdur) indicative of the start time ofthe shot. Once the shot has been cued and retrieved the video contentsof the shot can be checked on the preview screen 402 or a monitordirectly connected to the moving-image reproducing device. In addition,by pointing to and selecting an arbitrary icon from among displayedicons and setting the selected icon to a nondisplay state, it is alsopossible to eliminate an erroneously detected icon from the listdisplay. Reference numeral 412 denotes a panel for setting the thresholdshown in FIG. 3 as well as that used in a computer program which will bedescribed below.

The threshold th1 of FIG. 3 is displayed in an area 414. The area 414 isan interface which serves both a number inputting function and a numberdisplaying function. If the area 414 is pointed to with the cursor 400,the area 414 is changed to a mode for accepting a number input, so thatthe value of the threshold th1 can be changed by using an input devicesuch as a keyboard. Reference numeral 416 denotes a start button. If thestart button 416 is pointed to, processing for detecting a shot of shorttime duration is started. Reference numeral 418 denotes an end button.If the end button 418 is pointed to, the detecting processing is broughtto an end.

FIG. 5 illustrates in detail a process of displaying, in the form of alist, representative images in the screen area 406 shown in FIG. 4.Reference numeral 500 denotes an icon of a representative imagecorresponding to a detected shot of short time duration. The start timeof the shot is displayed together below the icon 500. In addition, asshown, the icon 500 is displayed as if several icons were stacked one ontop of another, to represent the time duration of the shot. Thus, thenumber of stacked icons represent the time duration of the shot, therebymaking it possible for a user to readily understand how long the shotlasts. The icon 500 also may be represented as a three-dimensionalrectangular parallelepiped having a front face which displays therepresentative image, and the time duration of the shot may berepresented as the thickness of the rectangular parallelepiped in thedepth direction thereof.

FIG. 6 illustrates in detail another technique for displayingrepresentative images in the form of a list. Reference numeral 602denotes an icon of a list of icons of representative imagescorresponding to detected shots of short time duration. The next andprevious shots or frames represented by icons 600 and 604, respectively,are displayed together on opposite sides of the icon 602. This techniquecan be realized by displaying the stored three representative images ina side-by-side manner, and by modifying the algorithm of FIG. 3 so thatthe number of representative images to be stored is increased by one.Increasing the number of representative images to be stored by onecauses the storage of a representative image corresponding to the pastshot change point. These three representative images are displayed asone set in the screen area 406 shown in FIG. 4. At this time, aplurality of representative images may be displayed in indented rowseach containing one set of three representative images, or threerepresentative images for each row may be displayed on the samebackground color, so that three representative images can be identifiedas one set.

The above-described procedure implemented by the computer program whenexecuted, the flowchart of which is illustrated in FIG. 3, finds itsgreatest use in detecting a shot of short time duration by detecting ashot change point preceding the shot of short time duration and a shotchange point succeeding the shot of short time duration. However, thetechnique of detecting shot change points using the above-describedprocedure has a tendency of detecting as a shot change point any pointsin the moving image wherein a difference between frames is high. Such ahigh difference between frames can occur, for example, in a part of amoving image having a quick change such as "dissolve" frames or "fadein/out" frames. When such a quick change part occurs in the movingimage, the above-described procedure erroneously detects that aplurality of shot change points have occurred. Using these erroneouslydetected continuously occurring shot change points, numerous shots ofshort duration are detected. Thus, the above-described procedure maydetect an excessive amount of shots of short time duration with a goodnumber of them being erroneous detections.

In the above-described technique it is also possible to detect a shot ofshort time duration corresponding to frames of poor picture quality anddisplay representative images of the frames of poor quality along withrepresentative images of frames containing subliminal suggestioninformation. Frames of poor picture quality can be detected since highdifferences between frames corresponding to shot changes points exist ina moving image having frames of poor picture quality. Representativeimages corresponding to the frames of poor picture quality can bedisplayed in the form of a list. Thereafter the user can quickly revieweach of the representative images in the list and determine whether therepresentative image corresponds to frames of poor picture quality orwhether the representative images corresponds to frames containingsubliminal suggestion information. Thus, frames of poor picture qualitycan be deleted from the moving image along with frames containingsubliminal suggestion information if desired.

Reviewing an excessive number of detected shots of short time durationand determining whether each of the shots contain subliminal suggestioninformation can be time consuming and tedious for a user. This isparticularly true when the detected shots of short time duration areplaced in the form of a list as illustrated in FIG. 6 for viewing laterby the user.

In order to address the above situation, the present invention providesan embodiment in which a hierarchical display of several detected shotsof short time duration are set forth in a set which is organized anddisplayed on a particular region of the display. Organizing severaldetected shots of short time duration into a hierarchical format with asingle representative image as one set reduces the number ofrepresentative images being displayed on the display screen.

The present invention provides a procedure for reducing the number ofdetected and displayed shots of short time duration where such detectedshots of short time duration may correspond to quick change parts suchas "dissolve" and "fade in/out" frames, frames of poor picture qualityor other such information not corresponding to subliminal suggestioninformation. By reducing the display of detected shots of short timeduration the tediousness and time consuming aspects of the review by theuser of each of the detected shots can be reduced.

The present invention accomplishes this by grouping detected shots ofshort time duration into a single group according to two conditions. Thefirst condition is whether the detected shots of short time durationoccur repeatedly within a very short time. The second condition iswhether a detected shot of short time duration occurs a short time aftera previously detected shot of short time duration, for example the lastshot of the previously formed group, and the frames of both shotscontain similar information.

Detected shots of short time duration that occur repeatedly within avery short time in most cases are an excessively detected shot of shorttime duration. In other words, each of the detected shots of short timeduration which occur within a short period of time may have occurred dueto, for example, frames of poor picture quality or frames where there isa quick change such as a "dissolve" scene or a "fade in/out" scene.Thus, grouping all of the detected shots of short time duration thatoccur within a very short period of time into a single group causes asingle representative image to be displayed in the list ofrepresentative images of detected shots of short time duration.

Accordingly, the user need not review representative images of each ofthe detected shots of short time duration which occur within a shortperiod of time since a single representative image of all of the shotsare included in the list. However, the display of the representativeimage of the detected shots of short time duration which occur within avery short period of time may be displayed in a hierarchical form toallow the user to individually review each of the detected shots ofshort time duration at a later time as per FIG. 7.

A detected shot of short time duration that occurs a very short timeafter another previously detected shot of short time duration of a groupand the frame of the shot has image information similar to the imageinformation contained in a frame of the previously detected shot ofshort time duration is a delayed detected shot of short time durationthat belong to the group of shots containing the previously detectedshot of short time duration. In other words, the delayed detected shotof short time duration which occurs within a short period of time fromthe last detected shot of a plurality of shots of short time durationthat have been grouped according to the above-described first condition,belong to such group if the frame of the detected shot of short timeduration has image information similar to the image information of theframe previously detected shot of short time duration. Thus, thedetected shot of short time duration that occur a very short time afteranother previously detected shot of short time duration and have imageinformation similar to the image information contained in the previouslydetected shot of short time duration is added to the group containingthe previously detected shot of short time duration.

Accordingly the user need not review the detected shot of short timeduration that occurs a very short period of time after anotherpreviously detected shot of short time duration and has imageinformation similar to the image information contained in the previouslydetected shot of short time duration. Since the delayed detected shot ofshort time duration is added to the group of the detected shots of shorttime duration containing the previously detected shot of short timeduration the user is only required to review the representative image ofthe group of detected shots of short time duration to which the delayeddetected shot of short time duration has been added. As described above,such representative image can be displayed in a hierarchial form toallow the user to individually review each of the detected shots ofshort time duration contained in the group represented by therepresentative image at a later time as shown in FIG. 7.

As described above, the hierarchical form used for checking therepresentative images is illustrated in FIG. 7. In FIG. 7 a window 700that displays the list of representative images corresponding to thedetected shots of short time duration. 410 is a scrolling bar. Thescrolling bar is used to scroll through a large number of representativeimages that cannot be displayed in a single display. 702 is an icon ofthe representative image of a detected shot of short time duration thathas been detected after the detection of a plurality of detected shotsof short time duration that have been grouped into a single grouprepresented by an icon 704 according to the above-described first andsecond conditions.

It is shown that two or more detected shots of short time duration makeone group which is represented using icon 704. When the user selectsicon 704 in the list display window 700 a window 706 that displays thedetected shots of short time duration that make up the group appears.

The details of the operation to be performed to open a window is notdescribed herein since such operation is well known due to thewidespread use of the Windows operating system of Microsoft and theoperating systems of the MacIntosh Computer of Apple Computers.

The flowchart of the computer program that groups a plurality of shotsof short time duration to a single group according to theabove-described first and second conditions is illustrated in FIG. 8.The computer program corresponding to the flowchart illustrated in FIG.8 is stored in memory 9.

The computer program when executed causes the computer 4 to operate asfollows. First the CPU 7 sets various variables to initial values thatare first necessary for execution of a program as an initializationprocess. (step 800). After the initialization process (step 800),initialization of the time just before shot change point (Tlast) isperformed (step 802). Then frame image (fn) outputted from amoving-image reproducing device 10 is accepted (step 804). Thereafter, adetection is performed to determine whether the frame is part of a shotof short time duration according to an already explained algorithm (step806).

When the frame is judged not to be a part of a shot of short timeduration, the processing proceeds back to step 804 where a new frame isaccepted (step 808). When the frame is judged to be part of a shot ofshort time duration, (step 808), the processing proceeds forward to(step 810).

In step 810 the following substeps are performed:

(1) The present time is substituted for "Tcur";

(2) The time "Tlast" of the time just before a shot of short timeduration is detected is subtracted from the time "Tcur" in order toobtain the time "Tdur" indicative of the length of time since the lastdetected shot of short time duration; and

(3) "Tcur" is substituted for "Tlast".

The values for the times described above can be provided by a clock,frame numbers that represent positions on the tape of a detected shot ofshort time duration or a time code. If frame numbers or time codes areused, the moving-image reproducing device 10 must be accessed for suchdata. If a time code in the form that is known as "VITC" and "LTC" isused, data corresponding to such code is read from the data that is sentin the signal line synchronizing with a frame.

The above-described first condition is satisfied, if it is determinedthat "Tdur" is shorter than a predetermined threshold "Th1" (step 812)causing the detected shot of short time duration to be added to the samegroup as the previously detected shot of short time duration (step 820).The best results for the first condition seem to occur when "Th1" is setto a value of approximately 500 milliseconds.

If the first condition is not satisfied a test for the second conditionis conducted. The second condition is satisfied if it is determined thata predetermined threshold "Th2" is larger than "Tdur" (step 814) and itis determined by comparison of the detected shot of short time durationto the previously detected shot of short time duration, that thedetected shots of short time duration resemble or are similar to thepreviously detected shot of short time duration (step 816) and thesimilarity is large (step 818). If the detected shot of short timeduration and the previously detected shot of short time durationresemble each and the similarity is large the detected shot of shorttime duration is added to the same group as the previously detected shotof short time duration (step 820). If the second condition is notsatisfied the processing proceeds back to step 804. The best results forthe second condition seem to occur when "Th2" is set to a value ofapproximately 5 seconds.

During the execution of the above-described computer program, the imageof the detected shots of short time duration of each group, thedetection times of the detected shots of short time duration of thegroup and a group identification (ID) corresponding to the group arestored in a particular structure in a storage area of memory 9 orexternal storage memory 6 in a manner illustrated in FIG. 9. FIG. 9illustrates the structure of the storage area including image data ofthe shots of short time duration grouped into a single group 900,detection times of the detected shots of short time duration of thegroup 902 and a group ID 906 for the group.

When a list is displayed on the screen, the image of a shot of shorttime duration and the detection time of the shot are read from memory 9or external storage device 6. In step 820 data of the detected shot ofshort time duration is added to the group of the previously detectedshot of short time duration according to the data contained in thestorage area 900, 902, 904 and is stored in memory 9 or external storagedevice 6.

The group ID 904 makes use of a serial number corresponding to thedetected shots of short time duration of the group. Thus, all detectedshots of short time duration having a same serial number correspondingto the serial number of the group ID belong to the same group.Therefore, when a group of shots of short time duration having the samegroup ID is displayed on a screen, for example, a shot having theearliest detected time is displayed as a representative image of thegroup. Also a symbol as illustrated in FIG. 7, indicating that therepresentative image representative of a plurality of detected shots ofshort time duration of a single set is shown simultaneously. When step820 is completed, processing returns to step 804, and theabove-described processing is repeated.

While the invention has been described in detail and pictorially in theaccompanying drawings it is not limited to such details since manychanges and modifications recognizable to those of ordinary skill in theart may be made to the invention without departing from the spirit andthe scope thereof.

We claim:
 1. A method of displaying a representative image correspondingto at least one shot of a duration within a specified range included ina plurality of shots of a moving image, said moving image includes aplurality of successive frames each having an image, and each shotincludes at least one frame, said method comprising the stepsof:sequentially detecting from said moving image a plurality of shotchange points representing points between a pair of frames where imagesof a preceding shot and images of a succeeding shot have changed;detecting whether a duration of a shot between a preceding shot changepoint and a succeeding shot change point is within the specified range;grouping shots having a duration within the specified range into a groupwhen said shots having a duration within the specified range repeatedlyoccur within a predetermined period of time due to frames where there isa quick change; and displaying a representative image corresponding tosaid group.
 2. A method of detecting a part of a moving image having aquick change, said moving image includes a plurality of successiveframes each having an image, said method comprising the stepsof:sequentially detecting from said moving image a plurality of shotchange points representing points between a pair of frames where imagesof a preceding shot and images of a succeeding shot have changed, eachshot including at least one frame; detecting whether a duration of ashot between a preceding shot change point and a succeeding shot changepoint is within the specified range; and grouping shots having aduration within the specified range into a group when said shots havinga duration within the specified range repeatedly occur within apredetermined period of time.
 3. A display apparatus for displaying arepresentative image corresponding to at least one shot of a durationwithin a specified range included in a plurality of shots of a movingimage, said moving image includes a plurality of successive frames eachhaving an image, and each shot includes at least one frame, saidapparatus comprising:a display having a display screen for displayingimages; and a processing apparatus for sequentially detecting from saidmoving image a plurality of change points representing points between apair of frames where images of a preceding shot and images of asucceeding shot have changes, detecting whether a duration of a shotbetween a preceding shot change point and a succeeding shot change pointis within the specified range, grouping shots having a duration withinthe specified range into a group when said shots having a durationwithin the specified range occur within a predetermined period of timedue to frames where there is quick change, and displaying on saiddisplay screen a representative image corresponding to said group.
 4. Adisplay apparatus for displaying a representative image corresponding toat least one shot of a duration within a specified range included in aplurality of shots of a moving image, said moving image includes aplurality of successive frames each having an image, and each shotincludes at least one frame, said apparatus comprising:a display havinga display screen for displaying images; and a processing apparatus forsequentially detecting from said moving image a plurality of shot changepoints representing points between a pair of frames where images of apreceding shot and images of a succeeding shot have changes and eachshot includes at least one frame, detecting whether a duration of a shotbetween a preceding shot change point and a succeeding shot change pointis within the specified range, and grouping shots having a durationwithin the specified range into a group when said shots having aduration within the specified range repeatedly occur within apredetermined period of time.
 5. A computer program stored on a computerreadable medium for execution by a computer to cause said computer todisplay a representative image corresponding to at least one shot of aduration within a specified range included in a plurality of shots of amoving image, said moving image includes a plurality of successiveframes each having an image, and each shot includes at least one frame,said computer program comprising:a first code for causing said computerto sequentially detect from said moving image a plurality of shot changepoints representing points between a pair of frames where images of apreceding shot and images of a succeeding shot have changes; a secondcode for causing said computer to detect whether a duration of a shotbetween a preceding shot change point and a succeeding shot change pointis within the specified range; a third code for causing said computer togroup shots having a duration within the specified range into a groupwhen said shots having a duration within the specified range repeatedlyoccur within a predetermined period of time due to frames where there isa quick change; and a fourth code for causing said computer to display arepresentative image corresponding to said group.
 6. A computer programstored in a computer readable medium for execution by a computer tocause said computer to display a representative image corresponding toat least one shot of a duration within a specified range included in aplurality of shots of a moving image, said moving image includes aplurality of successive frames each having an image, and each shotincludes at least one frame, said computer program comprising:a firstcode for causing said computer to sequentially detect from said movingimage a plurality of shot change points representing points between apair of frames where images of a preceding shot and images of asucceeding shot have changes, each shot including at least one frame; asecond code for causing said computer to detect whether a duration of ashot between a preceding shot change point and a succeeding shot changepoint is within the specified range; and a third code for causing saidcomputer to group shots having a duration within the specified rangeinto a group when said shots having a duration within the specifiedrange repeatedly occur within a predetermined period of time.